Apparatus, systems, and methods for local dimming in brightness-controlled environments

ABSTRACT

The disclosed display device may include (1) a display panel including pixel regions, (2) a backlight array coupled to the display panel that includes luminous elements, (3) a display housing configured to substantially prevent a user from referencing external brightness levels, (4) a display driver configured to receive an image including image blocks and scan the image to the display panel, and (5) a backlight driver configured to (a) determine an absolute brightness level of each of the image blocks, (b) derive, for each of the image blocks, a relative brightness level, (c) calculate, for each of the luminous elements, an illumination level based on the relative brightness level of a corresponding portion of the image blocks, and (d) illuminate, while the image is displayed via the display panel, each of the luminous elements according to the illumination level. Various other apparatus, systems, and methods are also disclosed.

BACKGROUND

Virtual reality (VR) and augmented reality (AR) headsets are gaining inpopularity for use in a growing number of activities. Such headsets mayintegrate visual information into a user's field of view to enhancetheir surroundings or allow them to step into immersivethree-dimensional environments. While virtual reality and augmentedreality headsets are often utilized for gaming and other entertainmentpurposes, they are also commonly employed for purposes outside ofrecreation-for example, governments may use them for military trainingsimulations, doctors may use them to practice surgery, and engineers mayuse them as visualization aids. Virtual and augmented reality systemsare also increasingly recognized for their utility in facilitatinginter-personal interactions between individuals in a variety ofcontexts.

Due to the compact size of many virtual and augmented reality headsets,display screens utilized in such headsets may need to have a smallprofile while also displaying high-quality, high-resolution images.Since a wearer's eyes may be positioned in relatively close proximity tothe display screen, which may be further magnified by lenses of theheadset, any inconsistencies in a displayed image may be more readilyapparent to a headset user than such inconsistencies in other types ofdisplay devices. Unfortunately, typical liquid-crystal displays (LCDs),which are sometimes integrated into headsets due to their comparativelylower cost and high availability, may exhibit certain undesirabledisplay artifacts. For example, conventional liquid crystal (LC) panelsare often prone to light leakage or “light bleed,” which may result inpoor contrast ratios and poor black levels. Some LCDs (e.g.,large-factor LCDs such as LCD televisions) may employ locally dimmablebacklight arrays to enhance contrast ratios and black levels, especiallywhen displaying high-contrast images. Unfortunately, conventional LCDsthat use local-dimming capable backlights typically exhibit haloingartifacts, especially around bright objects on darker backgrounds.Moreover, conventional backlights capable of local dimming typicallyhave slower refresh rates than the LC panels they illuminate, which mayexacerbate problems with display artifacts. For example, when displayedvia conventional LCDs that use local-dimming capable backlights, rapidlymoving objects may leave a ghosting trail in their wakes. As a result, auser's experience with conventional LCD headsets may be sub-optimal.

SUMMARY

As will be described in greater detail below, the instant disclosuredescribes various apparatus, systems, and methods for performing localdimming of backlights in brightness-controlled environments (e.g., VRheadsets where a user is substantially prevented from referencingexternal brightness levels). In some examples, a computer-implementedmethod may include (1) receiving an image including image blocks, (2)determining an absolute brightness level of each of the image blocks,(3) deriving, for each of the image blocks, a relative brightness levelbased on an internal reference brightness level, (4) calculating, foreach luminous element of a backlight array of a display panel, anillumination level based on the relative brightness level of acorresponding portion of the image blocks, and (5) illuminating, whilethe image is displayed via the display panel, each of the backlightarray's luminous elements according to the illumination level calculatedfor the luminous element. In some examples, the display panel mayinclude pixel regions, the backlight array may be coupled to the displaypanel behind the pixel regions and may include the luminous elementseach being configured to illuminate a corresponding portion of the pixelregions, and the display panel and the backlight array may be configuredto substantially prevent a viewer from referencing external brightnesslevels.

In some examples, the internal reference brightness level may includethe absolute brightness level or the relative brightness level ofanother one of the image blocks. In one example, the step of derivingthe relative brightness level for each of the image blocks may include(1) identifying a first image region of the image including one or moreof the image blocks having a lower absolute brightness level, (2)identifying a second image region of the image including one or more ofthe image blocks having a higher absolute brightness level, (3)calculating a difference between the lower absolute brightness level andthe higher absolute brightness level, (4) deriving, for each of theimage blocks in the first region, a first relative brightness level thatis lower than the lower absolute brightness level, and (5) deriving, foreach of the image blocks in the second region, a second relativebrightness level that is substantially equal to a sum of the firstrelative brightness level and the difference.

Additionally or alternatively, the step of deriving the relativebrightness level for each of the image blocks may include (1)identifying a first image region of the image having a lower absolutebrightness level, (2) identifying a second image region of the imageincluding two or more of the image blocks having a substantially similarhigher absolute brightness level, (3) deriving, for the second imageregion, a brightness level gradient that, when perceived by the viewer,substantially appears as a single brightness level, and (4) assigningbrightness levels from the brightness level gradient to the two or moreof the image blocks such that image blocks within the second imageregion furthest from the first image region have highest brightnesslevels and image blocks within the second image region closest to thefirst image region have lowest brightness levels. Additionally oralternatively, the step of deriving the relative brightness level foreach of the image blocks may include (1) identifying a first imageregion of the image having a higher absolute brightness level, (2)identifying a second image region of the image including two or more ofthe image blocks having a substantially similar lower absolutebrightness level, (3) deriving, for the second image region, abrightness level gradient that, when perceived by the viewer,substantially appears as a single brightness level, and (4) assigningbrightness levels from the brightness level gradient to the two or moreof the image blocks such that image blocks within the second imageregion furthest from the first image region have lowest brightnesslevels and image blocks within the second image region closest to thefirst image region have highest brightness levels.

In some examples, the internal reference brightness level may include anabsolute brightness level or a relative brightness level of an imageblock of an additional image previously displayed via the display panel.In some examples, the step of deriving the relative brightness level foreach of the image blocks may include (1) identifying an image block ofthe image having a first absolute brightness level, (2) identifying animage block of the additional image having a second absolute brightnesslevel substantially equal to the first absolute brightness level, (3)determining the relative brightness level of the image block of theadditional image, and (4) deriving a relative brightness level for theimage block of the image that is lower than the relative brightnesslevel of the image block of the additional image such that a differencebetween the relative brightness level of the image block of theadditional image and the relative brightness level of the image block ofthe image is substantially imperceptible to the viewer. In someexamples, the display panel and the backlight array may form a portionof a head-mounted display device, and the head-mounted display devicemay include a display housing surrounding the display panel and thebacklight array and configured to substantially prevent the viewer fromreferencing external brightness levels.

A computer-implemented method may include (1) receiving an image to bedisplayed via a display panel including pixel regions, (2) determiningan absolute brightness level of each image block of the image, (3) usinga model of human brightness perception to calculate, for each imageblock of the image, a relative brightness level based of the absolutebrightness level or the relative brightness level of another one of theimage blocks of the image or an absolute brightness level or a relativebrightness level of an image block of an additional image previouslydisplayed via the display panel, (4) calculating, for each luminouselement of a backlight array coupled to the display panel behind thepixel regions, an illumination level based on the relative brightnesslevel of a corresponding portion of the image blocks, and (5)illuminating, while the image is displayed via the display panel, eachof the luminous elements according to the illumination level calculatedfor the luminous element. In some examples, each of the backlightarray's luminous elements may be configured to illuminate acorresponding portion of the pixel regions, and the display panel andthe backlight array may be configured to substantially prevent a viewerfrom referencing external brightness levels. In some examples, the modelof human brightness perception may model how the viewer perceivesluminosity gradients. Additionally or alternatively, the model of humanbrightness perception may model how the viewer perceives absolutebrightness levels.

In some examples, the relative brightness level of one of the imageblocks may be calculated based on the absolute brightness level or therelative brightness level of another one of the image blocks. In oneexample, the step of using the model of human brightness perception tocalculate the relative brightness level for each of the image blocks mayinclude (1) identifying a first image region of the image including oneor more of the image blocks having a lower absolute brightness level,(2) identifying a second image region of the image including one or moreof the image blocks having a higher absolute brightness level, (3)calculating a difference between the lower absolute brightness level andthe higher absolute brightness level, (4) using the model of humanbrightness perception to derive, for each of the image blocks in thefirst region, a first relative brightness level that is lower than thelower absolute brightness level, and (5) deriving, for each of the imageblocks in the second region, a second relative brightness level that issubstantially equal to a sum of the first relative brightness level andthe difference.

In some examples, the step of using the model of human brightnessperception to calculate the relative brightness level for each of theimage blocks may include (1) identifying a first image region of theimage having a lower absolute brightness level, (2) identifying a secondimage region of the image including two or more of the image blockshaving a substantially similar higher absolute brightness level, (3)using the model of human brightness perception to derive, for the secondimage region, a brightness level gradient that, when perceived by theviewer, substantially appears as a single brightness level, and (4)assigning brightness levels from the brightness level gradient to thetwo or more of the image blocks such that image blocks within the secondimage region furthest from the first image region have highestbrightness levels and image blocks within the second image regionclosest to the first image region have lowest brightness levels.

In some examples, the step of using the model of human brightnessperception to calculate the relative brightness level for each of theimage blocks may include (1) identifying a first image region of theimage having a higher absolute brightness level, (2) identifying asecond image region of the image including two or more of the imageblocks having a substantially similar lower absolute brightness level,(3) using the model of human brightness perception to derive, for thesecond image region, a brightness level gradient that, when perceived bythe viewer, substantially appears as a single brightness level, and (4)assigning brightness levels from the brightness level gradient to thetwo or more of the image blocks such that image blocks within the secondimage region furthest from the first image region have lowest brightnesslevels and image blocks within the second image region closest to thefirst image region have highest brightness levels.

In some examples, the relative brightness level of one of the imageblocks may be calculated based on the absolute brightness level or therelative brightness level of an image block of an additional imagepreviously displayed via the display panel. In some examples, the stepof deriving the relative brightness level for each of the image blocksmay include (1) identifying image block of the image having a firstabsolute brightness level, (2) identifying image block of the additionalimage having a second absolute brightness level substantially equal tothe first absolute brightness level, (3) determining the relativebrightness level of the image block of the additional image, and (4)using the model of human brightness perception to derive a relativebrightness level for the image block of the image that is lower than therelative brightness level of the image block of the additional imagesuch that a difference between the relative brightness level of theimage block of the additional image and the relative brightness level ofthe image block of the image is substantially imperceptible to theviewer.

In addition, a corresponding display device may include (1) a displaypanel including pixel regions, (2) a backlight array coupled to thedisplay panel behind the pixel regions that includes luminous elementseach being configured to illuminate a corresponding one of the pixelregions, (3) a display housing surrounding the display panel and thebacklight array and configured to substantially prevent a user fromreferencing external brightness levels, (4) a display driver configuredto receive an image including image blocks and scan the image to thedisplay panel, and (5) a backlight driver configured to (a) determine anabsolute brightness level of each of the image blocks, (b) derive, foreach of the image blocks, a relative brightness level based of theabsolute brightness level or the relative brightness level of anotherone of the image blocks or an absolute brightness level or a relativebrightness level of an image block of an additional image previouslydisplayed via the display panel, (c) calculate, for each of the luminouselements, an illumination level based on the relative brightness levelof a corresponding portion of the image blocks, and (d) illuminate,while the image is displayed via the display panel, each of the luminouselements according to the illumination level calculated for the luminouselement. In some examples, the relative brightness levels of the imageblocks may be derived using a model of human brightness perceptionconfigured to model of how the user perceives luminosity gradients orhow the user perceives absolute brightness levels. In at least oneexample, the display device may include a head-mounted display device,and the display panel may include a liquid crystal panel. In someexamples, the display device may be a head-mounted display deviceconfigured to present an evolving three-dimensional virtual scene to theuser, and the image may depict elements in the evolvingthree-dimensional virtual scene. In such examples, the backlight drivermay be further configured to (1) determine a motion of the displaydevice, a head pose of the user relative to the evolvingthree-dimensional virtual scene, a gaze of the user, and/or one or moreof the elements, (2) calculate the illumination level for each of theplurality of luminous elements based on the motion, and/or (3) adjustthe image to compensate for the motion before displaying the image tothe user via the display panel.

Features from any of the above-mentioned embodiments may be used incombination with one another in accordance with the general principlesdescribed herein. These and other embodiments, features, and advantageswill be more fully understood upon reading the following detaileddescription in conjunction with the accompanying drawings and claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate a number of exemplary embodimentsand are a part of the specification. Together with the followingdescription, these drawings demonstrate and explain various principlesof the instant disclosure.

FIG. 1 is a block diagram of an exemplary display system in accordancewith some embodiments.

FIG. 2 is a perspective view of an exemplary head-mounted display systemin accordance with some embodiments.

FIG. 3 is a cross-sectional top view of an exemplaryhead-mounted-display device in accordance with some embodiments.

FIG. 4A is a front view of an exemplary head-mounted-display device inaccordance with some embodiments.

FIG. 4B is a front view of an exemplary LC panel in accordance with someembodiments.

FIG. 5A is a front view of an exemplary backlight array in accordancewith some embodiments.

FIG. 5B is a perspective view of a portion of the exemplary LC panelillustrated in FIG. 4B and a corresponding portion of the exemplarybacklight array illustrated in FIG. 5A in accordance with someembodiments.

FIG. 6 is a flow diagram of an exemplary method for performing localdimming of backlights in brightness-controlled environments inaccordance with some embodiments.

FIG. 7 is a flow diagram of an exemplary method for deriving relativebrightness levels in accordance with some embodiments.

FIG. 8A is a front view of an exemplary image in accordance with someembodiments.

FIG. 8B is a diagram of exemplary absolute and relative brightnesslevels corresponding to the exemplary image illustrated in FIG. 8A inaccordance with some embodiments.

FIG. 9 is a flow diagram of an exemplary method for deriving relativebrightness levels in accordance with some embodiments.

FIG. 10A is a front view of an additional exemplary image in accordancewith some embodiments.

FIG. 10B is a diagram of exemplary absolute and relative brightnesslevels corresponding to the additional exemplary image illustrated inFIG. 10A in accordance with some embodiments.

FIG. 11 is a diagram of additional exemplary absolute and relativebrightness levels corresponding to the additional exemplary imageillustrated in FIG. 10A in accordance with some embodiments.

FIG. 12 is a flow diagram of an exemplary method for deriving relativebrightness levels in accordance with some embodiments.

FIG. 13A is a front view of an additional exemplary image in accordancewith some embodiments.

FIG. 13B is a diagram of exemplary absolute and relative brightnesslevels corresponding to the additional exemplary image illustrated inFIG. 13A in accordance with some embodiments.

FIG. 14 is a flow diagram of an exemplary method for deriving relativebrightness levels in accordance with some embodiments.

FIG. 15A is a front view of an additional exemplary image in accordancewith some embodiments.

FIG. 15B is a diagram of exemplary absolute and relative brightnesslevels corresponding to the additional exemplary image illustrated inFIG. 15A in accordance with some embodiments.

FIG. 16 is a flow diagram of an additional exemplary method forperforming local dimming of backlights in brightness-controlledenvironments in accordance with some embodiments.

Throughout the drawings, identical reference characters and descriptionsindicate similar, but not necessarily identical, elements. While theexemplary embodiments described herein are susceptible to variousmodifications and alternative forms, specific embodiments have beenshown by way of example in the drawings and will be described in detailherein. However, the exemplary embodiments described herein are notintended to be limited to the particular forms disclosed. Rather, theinstant disclosure covers all modifications, equivalents, andalternatives falling within the scope of the appended claims.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

The present disclosure is generally directed to systems and methods forperforming local dimming in brightness-controlled environments and morespecifically directed to systems and methods for local backlight dimmingin VR headsets that utilize LCDs. In some examples, embodiments of theinstant disclosure may substantially prevent a viewer of a display panelfrom referencing brightness levels of external light sources (e.g.,brightness levels of light sources that do not illuminate the displaypanel) in order to illuminate the display panel based on relative,rather than absolute, internal brightness levels. In some examples, thedisclosed methods may model the specifics of how human eyes and brainswork to perceive absolute brightness levels to determine how toilluminate LCDs using various local dimming techniques. Since humanvision generally estimates absolute brightness based on relativebrightness, the systems and methods disclosed herein may make a portionof an image to appear bright by dimming its surroundings or make aportion of the image to appear dark by making its surroundings brighter.In addition, since human vision is generally able to compensate forillumination gradients and illumination strengths, the systems andmethods disclosed herein may use gradient backlighting techniques tovariably illuminate portions of an image with substantially the sameabsolute brightness levels in a way that is imperceptible to users. Insome examples, embodiments of the instant disclosure may leverage headposition, eye tracking, and/or object motion information available in VRheadsets to reduce potential local dimming visual artifacts that mightbe caused by head, eye, and object movement. By applying the disclosedlocal backlight dimming techniques to LCDs that fill a user's field ofview, the systems and methods disclosed herein may reduce or eliminatemany of the visual defects (e.g., static and/or temporal artifacts)found in LCDs that implement conventional local backlighting techniques.Moreover, the disclosed local backlight dimming techniques may enablecomfortable observation of fast-moving objects or bright objects on adark background, reduce power consumption of VR displays, and/orsignificantly increase the perceived contrast of VR scenes.

The following will provide, with reference to FIGS. 1-5B, examples ofhead-mounted display systems and devices. In addition, the discussioncorresponding to FIGS. 6-16 will provide examples of methods forperforming local dimming in brightness-controlled environments.

FIG. 1 is a block diagram of an exemplary display system 100 configuredto perform local dimming. As illustrated in this figure, example displaysystem 100 may include an LC panel 102, a backlight unit (BLU) 108, adisplay driver 114, a backlight driver 120, and a perception model 130.As shown in this example, LC panel 102 may include a left side 104 and aright side 106. Left side 104 and right side 106 may represent a leftportion and a right portion of pixel elements of LC panel 102,respectively. When incorporated in a head-mounted display system, leftside 104 and right side 106 may represent the portion of LC panel 102that is visible to a user's left eye and right eye, respectively. BLU108 may include a plurality of luminous elements or components thatgenerate and emit light. In some examples, BLU may include a leftbacklight 110 and a right backlight 112. Backlights 110 and 112 may eachinclude, for example, an array of luminous elements (e.g.,light-emitting diodes and/or laser emitting diodes).

Display driver 114 may include any suitable circuitry for driving pixelelements of LC panel 102, and backlight driver 120 may include anysuitable circuitry for controlling BLU 108. For example, display driver114 and/or backlight driver 120 may include at least one display driverintegrated circuit (IC). In some examples, display driver 114 mayinclude timing controller (TCON) circuitry that receives commands and/orimaging data and generates horizontal and vertical timing signals forthin-film-transistors (TFTs) of LC panel 102. In addition, backlightdriver 120 may include circuitry for generating timing andillumination-level signals for backlights 110 and 112. In someembodiments, display driver 114 may be mounted on an edge of a TFTsubstrate of LC panel 102 and electrically connected to scan lines anddata lines of LC panel 102. As illustrated in FIG. 1, display driver 114and backlight driver 120 may each include one or more modules forperforming one or more tasks. As will be explained in greater detailbelow, display driver 114 may include a receiving module 116 and ascanning module 118, and backlight driver 120 may include a determiningmodule 122, a deriving module 124, a calculating module 126, and anilluminating module 128. Although illustrated as separate elements, oneor more of the modules in FIG. 1 may represent portions of a singlemodule or application.

Example display system 100 in FIG. 1 may be implemented and/orconfigured in a variety of ways. For example, as shown in FIG. 2, all ora portion of example display system 100 may represent portions ofexample head-mounted display system 200. Additionally or alternatively,display system 100 may be utilized in and/or in conjunction with anysuitable electronic display device, such as, for example, a television,a computer monitor, a laptop monitor, a tablet device, a portabledevice, such as a cellular telephone (e.g., a smartphone), a wrist-watchdevice, a pendant device or other wearable or miniature device, a mediaplayer, a camera viewfinder, a gaming device, a navigation device,and/or any other type of device including an electronic display, withoutlimitation.

FIG. 2 is a perspective view of a head-mounted display system 200 inaccordance with some embodiments. In some embodiments, head-mounteddisplay system 200 may include a head-mounted-display device 202, afacial-interface system 208, a strap assembly 214, and audio subsystems216. A head-mounted-display device may include any type or form ofdisplay device or system that is worn on or about a user's head anddisplays visual content to the user. Head-mounted-display devices maydisplay content in any suitable manner, including via a display element(e.g., LC panel 102). Head-mounted-display devices may also displaycontent in one or more of various media formats. For example, ahead-mounted-display device may display video, photos, and/orcomputer-generated imagery (CGI). Head-mounted-display device 202 mayinclude a display housing 210 surrounding various components ofhead-mounted-display device 202, including lenses 204 and 205 andvarious electronic components, including LC panels and backlights s asdescribed herein. Display housing 210 may include a housing back surface212 and side surfaces surrounding the internal components, and anopening surrounding a viewing region 206 at a front side of displayhousing 210.

Head-mounted-display devices may provide diverse and distinctive userexperiences. Some head-mounted-display devices may providevirtual-reality experiences (i.e., they may display computer-generatedor pre-recorded content), while other head-mounted display devices mayprovide real-world experiences (i.e., they may display live imagery fromthe physical world). Head-mounted displays may also provide any mixtureof live and virtual content. For example, virtual content may beprojected onto the physical world (e.g., via optical or videosee-through), which may result in augmented reality or mixed realityexperiences. Head-mounted-display devices may be configured to bemounted to a user's head in a number of ways. Some head-mounted-displaydevices may be incorporated into glasses or visors. Otherhead-mounted-display devices may be incorporated into helmets, hats, orother headwear. Examples of head-mounted-display devices may includeOCULUS RIFT, GOOGLE GLASS, HTC VIVE, SAMSUNG GEAR, etc.

In some embodiments, facial-interface system 208 may be configured tocomfortably rest against a region of a user's face, including a regionsurrounding the user's eyes, when head-mounted display system 200 isworn by the user. In these embodiments, facial-interface system 208 mayinclude an interface cushion that is configured to rest against portionsof the user's face (e.g., at least a portion of the user's nasal, cheek,temple, and/or forehead facial regions). Facial-interface system 208 maysurround viewing region 206, which includes the user's field of vision,allowing the user to look through lenses 204 and 205 ofhead-mounted-display device 202 without interference from outside lightand without referencing external brightness levels while the user iswearing head-mounted display system 200. In at least one example,facial-interface system 208 may include one or more sensors configuredto monitor the users gaze (e.g., gaze direction, gaze origin, etc.).

In some embodiments, head-mounted display system 200 may include one ormore sensors that generate measurement signals in response to motion ofhead-mounted display system 200 (e.g., accelerometers, gyroscopes,magnetometers, other suitable types of sensors that detect motion, orsome combination thereof). In some examples, head-mounted display system200 may include a position sensor, an inertial measurement unit (IMU), adepth camera assembly, or any combination thereof. As noted, someartificial reality systems may, instead of blending an artificialreality with actual reality, substantially replace one or more of auser's sensory perceptions of the real world with a virtual experience.In some examples, head-mounted display system 200 may mostly orcompletely covers a user's field of view.

In some embodiments, head-mounted display system 200 may include varioustypes of computer vision components and subsystems. For example,head-mounted display system 200 may include one or more optical sensorssuch as two-dimensional (2D) or three-dimensional (3D) cameras,time-of-flight depth sensors, single-beam or sweeping laserrangefinders, 3D LiDAR sensors, and/or any other suitable type or formof optical sensor. Head-mounted display system 200 may process data fromone or more of these sensors to identify a location of a user, to mapthe real world, to provide a user with context about real-worldsurroundings, and/or to perform a variety of other functions (e.g.,determine changes to the orientation or position of head-mounted displaysystem 200, a head pose of the user wearing head-mounted display system200, or a gaze of the user).

In some embodiments, head-mounted display system 200 may map a user'senvironment and/or track motions of the user within the environmentusing techniques referred to as “simultaneous location and mapping”(SLAM). SLAM mapping and location identifying techniques may involve avariety of hardware and software tools that can create or update a mapof an environment while simultaneously keeping track of a user'slocation and/or orientation within the mapped environment. SLAM may usemany different types of sensors to create a map and determine a user'sposition within the map. SLAM techniques may, for example, implementoptical sensors to determine a user's location. Radios including WiFi,Bluetooth, global positioning system (GPS), cellular or othercommunication devices may also be used to determine a user's locationrelative to a radio transceiver or group of transceivers (e.g., a WiFirouter or group of GPS satellites). Head-mounted display system 200 mayincorporate any or all of these types of sensors to perform SLAMoperations such as creating and continually updating maps of a user'scurrent environment. In at least some of the embodiments describedherein, SLAM data generated by these sensors may be referred to as“environmental data” and may indicate a user's current environment. Thisdata may be stored in a local or remote data store (e.g., a cloud datastore) and may be provided to a user's AR/VR device on demand.

FIG. 3 shows an exemplary cross-sectional top view ofhead-mounted-display device 202. As shown in this figure, LC panel 102,BLU 108, display driver 114, and backlight driver 120 may be disposedwithin display housing 210 of head-mounted-display device 202. LC panel102 may be disposed within display housing 210 relative to lenses 204and 205 such that images produced by a display region of LC panel 102are visible to a user through lenses 204 and 205. As shown, LC panel 102may be positioned and oriented in display housing 210 such that a frontsurface of LC panel 102 faces towards lenses 204 and 205. As shown, leftbacklight 110 may be positioned behind left side 104 of LC panel 102,and right backlight 112 may be positioned behind right side 106 of LCpanel 102. As such, light 300 emitted from luminous elements of leftbacklight 110 through left side 104 of LC panel 102 may be visible to auser's left eye, and light 302 emitted from luminous elements of rightbacklight 112 through right side 106 of LC panel 102 may be visible tothe user's right eye. While not illustrated in FIG. 3, in someembodiments, a light diffuser may be sandwiched between LC panel 102 andBLU 108 in order to diffuse light 300 and light 302.

FIGS. 4A and 4B respectively show front views of head-mounted-displaydevice 202 and LC panel 102. As shown in FIG. 4A, head-mounted-displaydevice 202 may include at least one display, such as LC panel 102,disposed within display housing 210. In some embodiments, distinctportions of LC panel 102 may be visible to each of a user's eyes, withportions visible to each eye being separated by a dividing region 221(e.g., separate eye cups, a central partition, etc.) extending betweenlenses 204 and 205 and LC panel 102. Such a configuration may enabledistinct images to be presented by LC panel 102 to each of the user'seyes, allowing for 3-dimensional images to be perceived by the user.

As shown in FIG. 4A, head-mounted-display device 202 may also include alight-blocking panel 219 surrounding lenses 204 and 205. Light-blockingpanel 219 may, for example, extend between lenses 204 and 205 andsurrounding portions of display housing 210. Light-blocking panel 219may include, for example, a light-absorbing material (e.g., a darkpolymeric and/or fabric material) that masks internal components ofhead-mounted-display device 202 and that prevents any outside lightincidentally entering viewing region 206 (e.g., through a gap betweenthe user's face and facial-interface system 208) from being reflectedwithin viewing region 206. Display housing 210 may include a rigidmaterial, such as a rigid plastic, that supports and protects internalcomponents, such as LC panel 102, BLU 108, and other electronics.

As shown in FIG. 4B, LC panel 102 may include an M×N array of pixelelements (e.g., pixels and/or sub-pixels) that form visible imagesaccording to a suitable LCD technology (e.g., fast switching liquidcrystal technology). As shown, LC panel 102 may include M pixel-elementcolumns 402 and N pixel-element rows 400. Each pixel element of LC panel102 may include LC material that changes states (i.e., orientations ofliquid crystals) in response to applied currents or voltages. In someexamples, images may be displayed via LC panel 102 by driving pixelelements at different currents and/or voltages such that the pixelelements' LC material takes on different states and different amounts ofpolarization is given to light emitted through each of the pixelelements. A wide variety of visible colors may be produced by combiningdifferent amounts of light passed through sub-pixel color regions (e.g.,red, green, and/or blue color regions) of a color filter array panelsuch that a user perceives colors corresponding to the combinations ofthe sub-pixel colors.

In some embodiments, display driver 114 may display an image via LCpanel 102 by sending corresponding input signals to each of rows 400 ofLC panel 102, with the input signals being sequentially scanned alongrows 400 from row 0 to row N. These input signals may set LC material ateach of rows 400 to new states suitable for displaying the image.Backlight driver 120 may initiate an illumination of a portion of rows400 after its LC material has completely transitioned to the new statesas described below. For example, backlight driver 120 may initiate anillumination of backlight 110 to illuminate left side 104 after its LCmaterial has completely transitioned and may initiate an illumination ofbacklight 112 to illuminate right side 106 after its LC material hascompletely transitioned.

As shown in FIG. 5A, BLU 108 may include an M×N array of luminouselements 504 that each emit light at variable intensities. As shown, BLU108 may include M luminous-element columns 502 and N luminous-elementrows 500. As shown in FIG. 5B, each of luminous elements 504 of BLU 108may be configured to illuminate a corresponding zone 508 of the pixelelements of LC panel 102. In some examples, a light diffuser 506 may besandwiched between LC panel 102 and BLU 108 to diffuse the light emittedby a single luminous element 504 across its corresponding zone 508. Ingeneral, LC panel 102 may be a higher resolution panel, and BLU may be alower resolution panel.

FIG. 6 is a flow diagram of an example computer-implemented method 600for performing local dimming of backlights in brightness-controlledenvironments. The steps shown in FIG. 6 may be performed by any suitablecomputer-executable code and/or computing system, including displaysystem 100 in FIG. 1, head-mounted-display device 202 in FIG. 2, and/orvariations or combinations of one or more of the same. In one example,each of the steps shown in FIG. 6 may represent an algorithm whosestructure includes and/or is represented by multiple sub-steps, examplesof which will be provided in greater detail below.

As illustrated in FIG. 6, at step 602, one or more of the apparatus orsystems described herein may receive an image to be displayed via adisplay panel. For example, receiving module 116 may, as part of displaydriver 114, receive image 132 to be displayed via LC panel 102. Ingeneral, the apparatus or systems described herein may receive series ofimages (e.g., a sequence of video frames) to display to a user via adisplay panel.

At step 604, one or more of the apparatus or systems described hereinmay determine an absolute brightness level of each image block of theimage. For example, determining module 122 may, as part of backlightdriver 120, determine an absolute brightness level of each image blockof image 132. In some examples, the term “image block” may refer toindividual pixels of an image. In other examples, the term “image block”may refer to a portion of a display panel that is illuminated by asingle luminous element of a backlight array. In some examples, the term“absolute brightness level” may refer to a grayscale value, a lightnessvalue, or luminance value of any portion of an image. In some examples,the term “absolute brightness level” may refer to a grayscale value orluminance value of a pixel of an image. Additionally or alternatively,the term “absolute brightness level” may refer to a digitalrepresentation of relative luminance in a particular color space.

At step 606, one or more of the apparatus or systems described hereinmay derive, for each image block of the image, a relative brightnesslevel based on an internal reference brightness level. For example,deriving module 124 may, as part of backlight driver 120, derive, foreach image block of image 132, a relative brightness level based on aparticular predetermined internal reference brightness level. In someexamples, the term “relative brightness level” may refer to a brightnesslevel of any portion of an image that is relative to a referencebrightness level. In some examples, the term “relative brightness level”may refer to a brightness level perceived by a viewer (e.g., a perceivedgrayscale value, lightness value, or luminance value of a pixel of animage when viewed next to another reference grayscale value, lightnessvalue, or luminance value). In some examples, the apparatus or systemsdescribed herein may transform the image received at step 602 into acorrected image that is more suitable for display via a backlit LCDdisplay by adjusting the absolute brightness levels of the imagereceived at step 602 based on the derived relative brightness levels ofits image blocks such that the differences between the absolutebrightness levels of the image blocks of the image received at step 602and the absolute brightness levels of the corresponding image blocks ofthe corrected image are imperceptible to an observer. As such, one ormore users may be unable to tell the difference between the imagereceived at step 602 and the corrected image (e.g., if observed in aclosed environment without seeing other external objects).

The apparatus or systems described herein may use a variety of internalreference brightness levels to derive relative brightness levels.Examples of internal reference brightness levels may include, withoutlimitation, the absolute brightness level of one or more image blocks ofan image, the relative brightness level of another one of the imageblocks, the absolute brightness level of one or more image blocks of apreviously displayed image, and/or the relative brightness level of oneor more image blocks of a previously displayed image.

FIG. 7 illustrates an exemplary method for deriving relative brightnesslevels for pixels of image 132, illustrated in FIG. 8A. In this example,image 132 may include M pixel columns 802 and N pixel rows 800 and mayinclude a left side 804 for display via left side 104 of LC panel 102and a right side 806 for display via right side 106 of LC panel 102. Asshown in FIG. 7, at step 702, the apparatus or systems described hereinmay identify a first image region of an image including one or moreimage blocks having a lower absolute brightness level. For example,deriving module 124 may identify image region 810 of image 132illustrated in FIG. 8A that includes image blocks having an absolutebrightness level equal to AB1 shown in FIG. 8B, which illustratesbrightness levels 822 of the pixels in column 812 of image 132. In theexample shown in FIG. 8B, line 824 may represent the absolute brightnesslevels of pixels in column 812 of image 132, and line 826 may representderived relative brightness levels for pixels in column 812 of image132.

At step 704, the apparatus or systems described herein may identify asecond image region of the image including one or more of the imageblocks having a higher absolute brightness level. For example, derivingmodule 124 may identify image region 808 of image 132 illustrated inFIG. 8A that includes image blocks having an absolute brightness levelequal to AB2. At step 706, the apparatus or systems described herein maycalculate a difference between the lower absolute brightness level andthe higher absolute brightness level. For example, deriving module 124may calculate a difference 828 between AB1 and AB2. At step 708, theapparatus or systems described herein may derive, for each of the imageblocks in the first region, a first relative brightness level that islower than the lower absolute brightness level. For example, derivingmodule 124 may derive, for each of the image blocks in region 810, arelative brightness level that is equal to RB1. In general, theapparatus or systems described herein may choose a value for the firstrelative brightness level such that there is little to no perceptibledifference between the first relative brightness level and the lowerabsolute brightness level. At step 710, the apparatus or systemsdescribed herein may derive, for each of the image blocks in the secondregion, a second relative brightness level that is substantially equalto a sum of the first relative brightness level and the difference. Forexample, deriving module 124 may derive, for each of the image blocks inregion 808, a relative brightness level that is equal to RB2 (i.e., thesum of RB1 and difference 828).

FIG. 9 illustrates an exemplary method for deriving relative brightnesslevels for pixels of image 132 illustrated in FIG. 10A. In this example,image 132 may include M pixel columns 1002 and N pixel rows 1000 and mayinclude a left side 1004 for display via left side 104 of LC panel 102and a right side 1006 for display via right side 106 of LC panel 102. Asshown in FIG. 9, at step 902, the apparatus or systems described hereinmay identify a first image region of an image including one or moreimage blocks having a lower absolute brightness level. For example,deriving module 124 may identify image region 1010 of image 132illustrated in FIG. 10A that includes pixels having an absolutebrightness level equal to AB1 shown in FIG. 10B, which illustratesbrightness levels 1022 of the pixels in column 1012 of image 132. In theexample shown in FIG. 10B, line 1024 may represent the absolutebrightness levels of the pixels in column 1012 of image 132, and line1026 may represent derived relative brightness levels for the pixels incolumn 1012 of image 132.

At step 904, the apparatus or systems described herein may identify asecond image region of the image including one or more of the imageblocks having a higher absolute brightness level. For example, derivingmodule 124 may identify image region 1008 of image 132 illustrated inFIG. 10A that includes pixels having an absolute brightness level equalto AB2. At step 906, the apparatus or systems described herein mayderive, for the second image region, a brightness level gradient (e.g.,a series of brightness levels that gradually increase or decrease) that,when perceived by a viewer, substantially appears as a single brightnesslevel. For example, deriving module 124 may derive, for image region1008, a brightness level gradient 1028 and a brightness level gradient1030 that, when perceived by a viewer, substantially appears asbrightness level AB2. In another example, deriving module 124 mayderive, for image region 1008, a brightness level gradient 1102 and abrightness level gradient 1104 illustrated in FIG. 11 that, whenperceived by a viewer, substantially appears as brightness level AB2. Atstep 908, the apparatus or systems described herein may assignbrightness levels from the brightness level gradient to the two or moreof the image blocks such that image blocks within the second imageregion furthest from the first image region have highest brightnesslevels and image blocks within the second image region closest to thefirst image region have lowest brightness. For example, deriving module124 may assign brightness levels from brightness level gradients 1028and 1030 to pixels along column 1012 corresponding to image region 1008as shown such that pixels within image region 1008 furthest from imageregion 1010 have highest brightness levels and pixels within imageregion 1008 closest to image region 1010 have lowest brightness levels.

FIG. 12 illustrates an exemplary method for deriving relative brightnesslevels for pixels of image 132 illustrated in FIG. 13A. In this example,image 132 may include M pixel columns 1302 and N pixel rows 1300. Image132 may also include a left side 1304 for display via left side 104 ofLC panel 102 and a right side 1306 for display via right side 106 of LCpanel 102. As shown in FIG. 12, at step 1202, the apparatus or systemsdescribed herein may identify a first image region of an image having ahigher absolute brightness level. For example, deriving module 124 mayidentify image region 1308 of image 132 illustrated in FIG. 13A thatincludes pixels having an absolute brightness level equal to AB2 shownin FIG. 13B, which illustrates brightness levels 1322 of the pixels incolumn 1312 of image 132. In the example shown in FIG. 13B, line 1324may represent the absolute brightness levels of the pixels in column1312 of image 132, and line 1326 may represent derived relativebrightness levels for the pixels in column 1312 of image 132.

At step 1204, the apparatus or systems described herein may identify asecond image region of the image including two or more image blockshaving a substantially similar lower absolute brightness level. Forexample, deriving module 124 may identify image region 1310 of image 132illustrated in FIG. 13A that includes pixels having an absolutebrightness level equal to AB1. At step 1206, the apparatus or systemsdescribed herein may derive, for the second image region, a brightnesslevel gradient that, when perceived by a viewer, substantially appearsas a single brightness level. For example, deriving module 124 mayderive, for image region 1310, a brightness level gradient 1328 and abrightness level gradient 1330 that, when perceived by a viewer,substantially appears as brightness level AB1. At step 1208, theapparatus or systems described herein may assign brightness levels fromthe brightness level gradient to the two or more of the image blockssuch that image blocks within the second image region furthest from thefirst image region have lowest brightness levels and image blocks withinthe second image region closest to the first image region have highestbrightness levels. For example, deriving module 124 may assignbrightness levels from brightness level gradients 1328 and 1330 topixels along column 1312 corresponding to image region 1310 as shownsuch that pixels within image region 1310 furthest from image region1308 have lowest brightness levels and pixels within image region 1310closest to image region 1308 have highest brightness levels.

FIG. 14 illustrates an exemplary method for deriving relative brightnesslevels for pixels of image 134, illustrated in FIG. 15A. In thisexample, image 134 may represent a subsequent image displayed via LCpanel 102 that is substantially similar, at least in terms of absolutebrightness levels, to image 132. Like image 132, image 134 may include Mpixel columns 1502 and N pixel rows 1500 and may include a left side1504 for display via left side 104 of LC panel 102 and a right side 1506for display via right side 106 of LC panel 102. As shown in FIG. 14, atstep 1402, the apparatus or systems described herein may identify animage block of an image having a first absolute brightness level. Forexample, deriving module 124 may identify a pixel within image region1510 of image 134 illustrated in FIG. 15A having an absolute brightnesslevel equal to AB1 shown in FIG. 15B, which illustrates brightnesslevels 1522 of the pixels in column 1512 of image 134. In the exampleshown in FIG. 15B, line 1524 may represent the absolute brightnesslevels of the pixels in column 1512 of image 134 and pixels in column812 of image 132 in FIG. 8A, line 1526 may represent derived relativebrightness levels for the pixels in column 812 of image 132, and line1528 may represent derived relative brightness levels for the pixels incolumn 1512 of image 134.

At step 1404, the apparatus or systems described herein may identify animage block of an additional image having a second absolute brightnesslevel substantially equal to the first absolute brightness level. Forexample, deriving module 124 may identify a pixel within image region810 of image 132 illustrated in FIG. 8A having an absolute brightnesslevel equal to AB1 shown in FIG. 8B. At step 1406, the apparatus orsystems described herein may determine the relative brightness level ofthe image block of the additional image. For example, deriving module124 may determine that the relative brightness for the pixel withinimage region 810 of image 132 illustrated in FIG. 8A is equal to RB1shown in FIG. 8B.

At step 1408, the apparatus or systems described herein may derive arelative brightness level for the image block of the image that is lowerthan the relative brightness level of the image block of the additionalimage such that a difference between the relative brightness level ofthe image block of the additional image and the relative brightnesslevel of the image block of the image is substantially imperceptible tothe viewer. For example, deriving module 124 may derive a relativebrightness level equal to RB0 for the pixels within image region 1510 ofimage 134 illustrated in FIG. 15A such that a difference 1530 betweenRB1 and RB0 is substantially imperceptible to the viewer of image 134.

Returning to FIG. 6 at step 608, one or more of the apparatus or systemsdescribed herein may calculate, for each luminous element of a backlightof the display panel, an illumination level based on the relativebrightness level of a corresponding portion of the image blocks of theimage. For example, calculating module 126 may, as part of backlightdriver 120, calculate an illumination level for luminous element 504 ofBLU 108 based on the relative brightness levels of portion 508 of image132. In some examples, the calculated illumination levels may be derivedfrom the corrected image described above.

In some examples, one or more of the apparatus or systems describedherein may include a display device configured to present an evolvingthree-dimensional virtual scene to a user. In these examples, theapparatus or systems described herein may, in addition to or as analternative to calculating illumination levels based on relativebrightness levels, calculate illumination levels based on motions of thedisplay device, motions of a head pose of a user relative to theevolving three-dimensional virtual scene, motions of the user's gaze,and/or motions of one or more of the elements in the evolvingthree-dimensional virtual scene. In some examples, movements of adisplay device that displays an evolving three-dimensional virtualscene, movements of the head pose of a user relative to the evolvingthree-dimensional virtual scene, movements of a user's gaze, and/ormovements of the elements in the evolving three-dimensional virtualscene (which may be detectable using derived 2D motion vectors and/ordepth map information) may cause predictable movements of one or more ofthe elements in the images portraying the evolving three-dimensionalvirtual scene. As such, one or more of the apparatus or systemsdescribed herein may predict movements of one or more of the elements inthe images portraying the evolving three-dimensional virtual scene inorder to correctly calculate illumination levels for illuminating theseelements in the current image or in subsequent images. For example, theapparatus or systems described herein may use eye and object motionknowledge to predict the position of light regions in a subsequent image(N+1) more accurately in order to yield a more precise/aggressiveLED+LCD adjustment. In at least one example, by considering movementswithin the images displayed via a display device, the apparatus orsystems described herein may calculate illumination levels in ways thatreduce or eliminate temporal backlighting artifacts that would otherwisebe caused by the movements (e.g., a ghosting trail caused by a quicklymoving bright object).

At step 610, one or more of the apparatus or systems described hereinmay illuminate, while the image is displayed via the display panel, eachof the backlight array's luminous elements according to the illuminationlevel calculated for the luminous element. For example, illuminatingmodule 128 may, as part of backlight driver 120, illuminate, while image132 is displayed via LC panel 102, each of luminous elements 504according to the illumination level calculated at step 608. In someexamples, a backlight array may have far fewer luminous elements thanthe number of image blocks of the images that it illuminates. As such,each of the backlight array's luminous elements, which may have a singleillumination level, may need to illuminate multiple image blocks of animage that each have a different brightness level. For at least thisreason, one or more of the apparatus or systems described herein maycompensate for these differences before displaying and illuminating animage. For example, one or more of the apparatus or systems describedherein may transform the corrected image described above into a finalimage ready to be displayed and backlit by determining an image that,when illuminated according to the illumination levels calculated above,will produce an optical density field equivalent to an optical densityfield of the corrected image if illuminated by a uniformly illuminatedbacklight array.

In some examples, one or more of the apparatus or systems describedherein may include a display device configured to present imagesportraying an evolving three-dimensional virtual scene to a user. Insuch examples, movements of the display device, movements of the headpose of a user relative to the evolving three-dimensional virtual scene,movements of a user's gaze, and/or movements of the elements in theevolving three-dimensional virtual scene may cause predictable movementsof one or more of the elements in the images. As such, one or more ofthe apparatus or systems described herein may transform the images (orcorrected images) into a final image ready to be displayed and backlitby predicting the movements of one or more of the elements in the imagesportraying the evolving three-dimensional virtual scene in order tocompensate for the motions of the elements. In at least one example, byconsidering movements within the images displayed via a display device,the apparatus or systems described herein may transform the images inways that reduce or eliminate temporal backlighting artifacts that wouldotherwise be caused by the movements (e.g., a ghosting trail caused by aquickly moving bright object).

FIG. 16 is a flow diagram of an example computer-implemented method 1600for performing local dimming of backlights in brightness-controlledenvironments. The steps shown in FIG. 16 may be performed by anysuitable computer-executable code and/or computing system, includingdisplay system 100 in FIG. 1, head-mounted-display device 202 in FIG. 2,and/or variations or combinations of one or more of the same. In oneexample, each of the steps shown in FIG. 16 may represent an algorithmwhose structure includes and/or is represented by multiple sub-steps,examples of which will be provided in greater detail below.

As illustrated in FIG. 16, at step 1602, one or more of the apparatus orsystems described herein may receive an image to be displayed via adisplay panel. For example, receiving module 116 may, as part of displaydriver 114, receive image 132 to be displayed via LC panel 102. At step1604, one or more of the apparatus or systems described herein maydetermine an absolute brightness level of each image block of the image.For example, determining module 122 may, as part of backlight driver120, determine an absolute brightness level of each image block of image132.

At step 1606, one or more of the apparatus or systems described hereinmay use a model of human brightness perception to calculate, for each ofthe image blocks, a relative brightness level. For example, derivingmodule 124 may, as part of backlight driver 120, use perception model130 to derive, for each image block of image 132, a relative brightnesslevel. In some examples, the term “model of human brightness perception”may refer to any algorithm, heuristic, data, or combination thereof,that may be used to calculate relative brightness levels for imageblocks of an image based of the absolute brightness levels or therelative brightness levels of the image blocks of the image or theabsolute brightness levels or the relative brightness levels of imageblocks of a previously displayed image. In some examples, one or more ofthe apparatus or systems described herein may empirically create a modelof human brightness perception by comparing two sets of images that oneor more users were unable to tell the difference between (e.g., whenobserved in a closed environment without seeing other external objects).Additionally or alternatively, one or more of the apparatus or systemsdescribed herein may create a model of human brightness perception basedon scientific observations about human eye and/or brain biology (e.g.,observations regarding how the human vision system may be insensitive toor able to compensate for a wide range of brightness gradients on brightobjects).

In some examples, one or more of the apparatus or systems describedherein may train a model of human brightness perception to model orpredict how a viewer perceives luminosity gradients. In these examples,one or more of the apparatus or systems described herein may use themodel to derive the brightness level gradients described above that,when perceived by the viewer, substantially appears as a singlebrightness level. Additionally or alternatively, one or more of theapparatus or systems described herein may train a model of humanbrightness perception to model or predict how a viewer perceivesabsolute brightness levels based on reference brightness levels. In someexamples, one or more of the apparatus or systems described herein mayuse the model to derive the differences between brightness levelsdescribed above that are substantially imperceptible to a viewer.

At step 1608, one or more of the apparatus or systems described hereinmay calculate, for each luminous element of a backlight of the displaypanel, an illumination level based on the relative brightness level of acorresponding portion of the image blocks of the image. For example,calculating module 126 may, as part of backlight driver 120, calculatean illumination level for luminous element 504 of BLU 108 based on therelative brightness levels of corresponding portion 508 of image 132. Atstep 1610, one or more of the apparatus or systems described herein mayilluminate, while the image is displayed via the display panel, each ofthe backlight array's luminous elements according to the illuminationlevel calculated for the luminous element. For example, illuminatingmodule 128 may, as part of backlight driver 120, illuminate, while image132 is displayed via LC panel 102, each of luminous elements 504according to the illumination level calculated at step 1608.

As discussed throughout the instant disclosure, the disclosedapparatuses, systems, and methods may provide one or more advantagesover traditional display apparatuses, systems, and methods. For example,embodiments of the instant disclosure may reduce or eliminate many ofthe visual defects found in LCDs that implement conventional localbacklighting techniques. Moreover, the disclosed local backlight dimmingtechniques may enable comfortable observation of fast-moving objects orbright objects on a dark background, reduce power consumption of VRdisplays, and/or significantly increase the perceived contrast of VRscenes.

As detailed above, the computing devices and systems described and/orillustrated herein broadly represent any type or form of computingdevice or system capable of executing computer-readable instructions,such as those contained within the modules described herein. In theirmost basic configuration, these computing device(s) may each include atleast one memory device and at least one physical processor.

In some examples, the term “memory device” generally refers to any typeor form of volatile or non-volatile storage device or medium capable ofstoring data and/or computer-readable instructions. In one example, amemory device may store, load, and/or maintain one or more of themodules described herein. Examples of memory devices include, withoutlimitation, Random Access Memory (RAM), Read Only Memory (ROM), flashmemory, Hard Disk Drives (HDDs), Solid-State Drives (SSDs), optical diskdrives, caches, variations or combinations of one or more of the same,or any other suitable storage memory.

In some examples, the term “physical processor” generally refers to anytype or form of hardware-implemented processing unit capable ofinterpreting and/or executing computer-readable instructions. In oneexample, a physical processor may access and/or modify one or moremodules stored in the above-described memory device. Examples ofphysical processors include, without limitation, microprocessors,microcontrollers, Central Processing Units (CPUs), Field-ProgrammableGate Arrays (FPGAs) that implement softcore processors,Application-Specific Integrated Circuits (ASICs), portions of one ormore of the same, variations or combinations of one or more of the same,or any other suitable physical processor.

Although illustrated as separate elements, the modules described and/orillustrated herein may represent portions of a single module orapplication. In addition, in certain embodiments one or more of thesemodules may represent one or more software applications or programsthat, when executed by a computing device, may cause the computingdevice to perform one or more tasks. For example, one or more of themodules described and/or illustrated herein may represent modules storedand configured to run on one or more of the computing devices or systemsdescribed and/or illustrated herein. One or more of these modules mayalso represent all or portions of one or more special-purpose computersconfigured to perform one or more tasks.

In addition, one or more of the modules described herein may transformdata, physical devices, and/or representations of physical devices fromone form to another. For example, one or more of the modules recitedherein may receive an image to be displayed via a LC panel, transformthe image into illumination levels for luminous elements of the LCpanel's backlight array, output a result of the transformation to thebacklight array of the LC panel, and use the result of thetransformation to illuminate the image while the image is displayed viathe LC panel. Additionally or alternatively, one or more of the modulesrecited herein may transform a processor, volatile memory, non-volatilememory, and/or any other portion of a physical computing device from oneform to another by executing on the computing device, storing data onthe computing device, and/or otherwise interacting with the computingdevice.

In some embodiments, the term “computer-readable medium” generallyrefers to any form of device, carrier, or medium capable of storing orcarrying computer-readable instructions. Examples of computer-readablemedia include, without limitation, transmission-type media, such ascarrier waves, and non-transitory-type media, such as magnetic-storagemedia (e.g., hard disk drives, tape drives, and floppy disks),optical-storage media (e.g., Compact Disks (CDs), Digital Video Disks(DVDs), and BLU-RAY disks), electronic-storage media (e.g., solid-statedrives and flash media), and other distribution systems.

Embodiments of the instant disclosure may include or be implemented inconjunction with an artificial reality system. Artificial reality is aform of reality that has been adjusted in some manner beforepresentation to a user, which may include, e.g., a virtual reality (VR),an augmented reality (AR), a mixed reality (MR), a hybrid reality, orsome combination and/or derivatives thereof. Artificial reality contentmay include completely generated content or generated content combinedwith captured (e.g., real-world) content. The artificial reality contentmay include video, audio, haptic feedback, or some combination thereof,any of which may be presented in a single channel or in multiplechannels (such as stereo video that produces a three-dimensional effectto the viewer). Additionally, in some embodiments, artificial realitymay also be associated with applications, products, accessories,services, or some combination thereof, that are used to, e.g., createcontent in an artificial reality and/or are otherwise used in (e.g.,perform activities in) an artificial reality. The artificial realitysystem that provides the artificial reality content may be implementedon various platforms, including a head-mounted display (HMD) connectedto a host computer system, a standalone HMD, a mobile device orcomputing system, or any other hardware platform capable of providingartificial reality content to one or more viewers.

The process parameters and sequence of the steps described and/orillustrated herein are given by way of example only and can be varied asdesired. For example, while the steps illustrated and/or describedherein may be shown or discussed in a particular order, these steps donot necessarily need to be performed in the order illustrated ordiscussed. The various exemplary methods described and/or illustratedherein may also omit one or more of the steps described or illustratedherein or include additional steps in addition to those disclosed.

The preceding description has been provided to enable others skilled inthe art to best utilize various aspects of the exemplary embodimentsdisclosed herein. This exemplary description is not intended to beexhaustive or to be limited to any precise form disclosed. Manymodifications and variations are possible without departing from thespirit and scope of the instant disclosure. The embodiments disclosedherein should be considered in all respects illustrative and notrestrictive. Reference should be made to the appended claims and theirequivalents in determining the scope of the instant disclosure.

Unless otherwise noted, the terms “connected to” and “coupled to” (andtheir derivatives), as used in the specification and claims, are to beconstrued as permitting both direct and indirect (i.e., via otherelements or components) connection. In addition, the terms “a” or “an,”as used in the specification and claims, are to be construed as meaning“at least one of.” Finally, for ease of use, the terms “including” and“having” (and their derivatives), as used in the specification andclaims, are interchangeable with and have the same meaning as the word“comprising.”

What is claimed is:
 1. A computer-implemented method comprising:receiving, at a head-mounted display device, an image comprising aplurality of image blocks, wherein: the image is to be displayed via adisplay panel of the head-mounted display device comprising a pluralityof pixel regions; a backlight array of the head-mounted display deviceis coupled to the display panel behind the plurality of pixel regionsand comprises a plurality of luminous elements that each illuminate acorresponding portion of the plurality of pixel regions; and a displayhousing of the head-mounted display device substantially prevents aviewer from seeing ambient brightness levels of the viewer'ssurroundings; determining, at the head-mounted display device, anabsolute brightness level of each of the plurality of image blocks;deriving, at the head-mounted display device for each of the pluralityof image blocks, a relative brightness level based on an internalreference brightness level instead of an ambient brightness level of theviewer's surroundings; calculating, at the head-mounted display devicefor each of the plurality of luminous elements, an illumination levelbased at least in part on the relative brightness level of acorresponding portion of the plurality of image blocks; andilluminating, at the head-mounted display device while displaying theimage via the display panel, each of the plurality of luminous elementsaccording to the illumination level calculated for the luminous element.2. The computer-implemented method of claim 1, wherein the internalreference brightness level comprises at least one of the absolutebrightness level or the relative brightness level of another one of theplurality of image blocks.
 3. The computer-implemented method of claim2, wherein deriving the relative brightness level for each of theplurality of image blocks comprises: identifying a first image region ofthe image comprising one or more of the plurality of image blocks havinga lower absolute brightness level; identifying a second image region ofthe image comprising one or more of the plurality of image blocks havinga higher absolute brightness level; calculating a difference between thelower absolute brightness level and the higher absolute brightnesslevel; deriving, for each of the plurality of image blocks in the firstregion, a first relative brightness level that is lower than the lowerabsolute brightness level; and deriving, for each of the plurality ofimage blocks in the second region, a second relative brightness levelthat is substantially equal to a sum of the first relative brightnesslevel and the difference.
 4. The computer-implemented method of claim 2,wherein deriving the relative brightness level for each of the pluralityof image blocks comprises: identifying a first image region of the imagehaving a lower absolute brightness level; identifying a second imageregion of the image comprising two or more of the plurality of imageblocks having a substantially similar higher absolute brightness level;deriving, for the second image region, a brightness level gradient that,when perceived by the viewer, substantially appears as a singlebrightness level; and assigning brightness levels from the brightnesslevel gradient to the two or more of the plurality of image blocks suchthat image blocks within the second image region furthest from the firstimage region have highest brightness levels and image blocks within thesecond image region closest to the first image region have lowestbrightness levels.
 5. The computer-implemented method of claim 2,wherein deriving the relative brightness level for each of the pluralityof image blocks comprises: identifying a first image region of the imagehaving a higher absolute brightness level; identifying a second imageregion of the image comprising two or more of the plurality of imageblocks having a substantially similar lower absolute brightness level;deriving, for the second image region, a brightness level gradient that,when perceived by the viewer, substantially appears as a singlebrightness level; and assigning brightness levels from the brightnesslevel gradient to the two or more of the plurality of image blocks suchthat image blocks within the second image region furthest from the firstimage region have lowest brightness levels and image blocks within thesecond image region closest to the first image region have highestbrightness levels.
 6. The computer-implemented method of claim 1,wherein the internal reference brightness level comprises at least oneof an absolute brightness level or a relative brightness level of animage block of an additional image previously displayed via the displaypanel.
 7. The computer-implemented method of claim 6, wherein derivingthe relative brightness level for each of the plurality of image blockscomprises: identifying at least one image block of the image having afirst absolute brightness level; identifying at least one image block ofthe additional image having a second absolute brightness levelsubstantially equal to the first absolute brightness level; determiningthe relative brightness level of the at least one image block of theadditional image; and deriving a relative brightness level for the atleast one image block of the image that is lower than the relativebrightness level of the at least one image block of the additionalimage, a difference between the relative brightness level of the atleast one image block of the additional image and the relativebrightness level of the at least one image block of the image beingsubstantially imperceptible to the viewer.
 8. The computer-implementedmethod of claim 1, wherein the image conveys an evolvingthree-dimensional virtual scene to the viewer.
 9. A computer-implementedmethod comprising: receiving, at a head-mounted display device, an imagecomprising a plurality of image blocks, wherein: the image is to bedisplayed via a display panel of the head-mounted display devicecomprising a plurality of pixel regions; a backlight array of thehead-mounted display device is coupled to the display panel behind theplurality of pixel regions and comprises a plurality of luminouselements that each illuminate a corresponding portion of the pluralityof pixel regions; and a display housing of the head-mounted displaydevice substantially prevents a viewer from seeing ambient brightnesslevels of the viewer's surroundings; determining, at the head-mounteddisplay device, an absolute brightness level of each of the plurality ofimage blocks; using, at the head-mounted display device, a model ofhuman brightness perception to calculate, for each of the plurality ofimage blocks, a relative brightness level based on an internal referencebrightness level instead of an ambient brightness level of the viewer'ssurroundings, the internal reference brightness level comprising atleast one of: the absolute brightness level or the relative brightnesslevel of another one of the plurality of image blocks; or an absolutebrightness level or a relative brightness level of an image block of anadditional image previously displayed via the display panel;calculating, at the head-mounted display device for each of theplurality of luminous elements, an illumination level based at least inpart on the relative brightness level of a corresponding portion of theplurality of image blocks; and illuminating, at the head-mounted displaydevice while displaying the image via the display panel, each of theplurality of luminous elements according to the illumination levelcalculated for the luminous element.
 10. The computer-implemented methodof claim 9, wherein the model of human brightness perception predictshow the viewer perceives luminosity gradients when the viewer cannotcompare the internal reference brightness level to the ambientbrightness levels of the viewer's surroundings.
 11. Thecomputer-implemented method of claim 9, wherein the model of humanbrightness perception predicts how the viewer perceives absolutebrightness levels when the viewer cannot compare the internal referencebrightness level to the ambient brightness levels of the viewer'ssurroundings.
 12. The computer-implemented method of claim 9, whereinthe relative brightness level of at least one of the plurality of imageblocks is calculated based on at least one of the absolute brightnesslevel or the relative brightness level of the other one of the pluralityof image blocks.
 13. The computer-implemented method of claim 12,wherein using the model of human brightness perception to calculate therelative brightness level for each of the plurality of image blockscomprises: identifying a first image region of the image comprising oneor more of the plurality of image blocks having a lower absolutebrightness level; identifying a second image region of the imagecomprising one or more of the plurality of image blocks having a higherabsolute brightness level; calculating a difference between the lowerabsolute brightness level and the higher absolute brightness level;using the model of human brightness perception to derive, for each ofthe plurality of image blocks in the first region, a first relativebrightness level that is lower than the lower absolute brightness level;and deriving, for each of the plurality of image blocks in the secondregion, a second relative brightness level that is substantially equalto a sum of the first relative brightness level and the difference. 14.The computer-implemented method of claim 12, wherein using the model ofhuman brightness perception to calculate the relative brightness levelfor each of the plurality of image blocks comprises: identifying a firstimage region of the image having a lower absolute brightness level;identifying a second image region of the image comprising two or more ofthe plurality of image blocks having a substantially similar higherabsolute brightness level; using the model of human brightnessperception to derive, for the second image region, a brightness levelgradient that, when perceived by the viewer, substantially appears as asingle brightness level; and assigning brightness levels from thebrightness level gradient to the two or more of the plurality of imageblocks such that image blocks within the second image region furthestfrom the first image region have highest brightness levels and imageblocks within the second image region closest to the first image regionhave lowest brightness levels.
 15. The computer-implemented method ofclaim 12, wherein using the model of human brightness perception tocalculate the relative brightness level for each of the plurality ofimage blocks comprises: identifying a first image region of the imagehaving a higher absolute brightness level; identifying a second imageregion of the image comprising two or more of the plurality of imageblocks having a substantially similar lower absolute brightness level;using the model of human brightness perception to derive, for the secondimage region, a brightness level gradient that, when perceived by theviewer, substantially appears as a single brightness level; andassigning brightness levels from the brightness level gradient to thetwo or more of the plurality of image blocks such that image blockswithin the second image region furthest from the first image region havelowest brightness levels and image blocks within the second image regionclosest to the first image region have highest brightness levels. 16.The computer-implemented method of claim 9, wherein the relativebrightness level of at least one of the plurality of image blocks iscalculated based on at least one of the absolute brightness level or therelative brightness level of the image block of the additional imagepreviously displayed via the display panel.
 17. The computer-implementedmethod of claim 16, wherein deriving the relative brightness level foreach of the plurality of image blocks comprises: identifying at leastone image block of the image having a first absolute brightness level;identifying at least one image block of the additional image having asecond absolute brightness level substantially equal to the firstabsolute brightness level; determining the relative brightness level ofthe at least one image block of the additional image; and using themodel of human brightness perception to derive a relative brightnesslevel for the at least one image block of the image that is lower thanthe relative brightness level of the at least one image block of theadditional image, a difference between the relative brightness level ofthe at least one image block of the additional image and the relativebrightness level of the at least one image block of the image beingsubstantially imperceptible to the viewer.
 18. A head-mounted displaydevice comprising: a display panel comprising a plurality of pixelregions; a backlight array coupled to the display panel behind theplurality of pixel regions, the backlight array comprising a pluralityof luminous elements that each illuminate a corresponding one of theplurality of pixel regions; a display housing that substantiallyprevents a user from seeing ambient brightness levels; a display driverconfigured to: receive an image comprising a plurality of image blocks;and scan the image to the display panel; and a backlight driverconfigured to: determine an absolute brightness level of each of theplurality of image blocks; derive, for each of the plurality of imageblocks, a relative brightness level based on an internal referencebrightness level instead of an ambient brightness level of the user'ssurroundings, the internal reference brightness level comprising atleast one of: the absolute brightness level or the relative brightnesslevel of another one of the plurality of image blocks; or an absolutebrightness level or a relative brightness level of an image block of anadditional image previously displayed via the display panel; calculate,for each of the plurality of luminous elements, an illumination levelbased at least in part on the relative brightness level of acorresponding portion of the plurality of image blocks; and illuminate,while the image is displayed via the display panel, each of theplurality of luminous elements according to the illumination levelcalculated for the luminous element.
 19. The display device of claim 18,wherein: the head-mounted display device is configured to present anevolving three-dimensional virtual scene to the user; the image depictsa plurality of elements in the evolving three-dimensional virtual scene;and the backlight driver is further configured to: determine a motion ofat least one of: the display device; a head pose of the user relative tothe evolving three-dimensional virtual scene; a gaze of the user; or oneor more of the plurality of elements; and calculate the illuminationlevel for each of the plurality of luminous elements based at least inpart on the motion.
 20. The display device of claim 18, wherein: thehead-mounted display device is configured to present an evolvingthree-dimensional virtual scene to the user; the image depicts aplurality of elements in the evolving three-dimensional virtual scene;and the backlight driver is further configured to: determine a motion ofat least one of: the display device; a head pose of the user relative tothe evolving three-dimensional virtual scene; a gaze of the user; or oneor more of the plurality of elements; and adjust the image to compensatefor the motion before displaying the image to the user via the displaypanel.